THIS APPLICATION IS A U.S. NATIONAL PHASE APPLICATION OF PCT INTERNATIONAL APPLICATION PCT/JP97/00117.
1. Field of the Invention
The present invention relates to a method of padding a digital picture having an arbitrary shape, and an encoder and a decoder of digital picture using the same method.
2. Background Art
It is necessary to compress (encode) a digital picture for promoting the efficiency of its storage and transmission. Several methods of encoding are available as prior arts such as xe2x80x9cdiscrete cosine transformxe2x80x9d (DCT) including JPEG and MPEG, and other wave-form encoding methods such as xe2x80x9csubbandxe2x80x9d, xe2x80x9cwaveletxe2x80x9d, xe2x80x9cfractalxe2x80x9d and the like. Further, in order to remove a redundant signal between pictures, a prediction method between pictures is employed, and then the differential signal is encoded by wave-form encoding method.
According to the recent trend, the object constituting a picture are individually encoded and transmitted, for improving the coding efficiency as well as allowing reproduction of the individual objects which constitute a picture. On a reproducing side, each object is decoded, and the reproduced objects are composited into the picture for displaying. Per-object base encoding method allows the user to combine objects arbitrarily, whereby a motion picture can be re-edited with ease. Further, depending on the congestion of the communication channel, performance of a reproducing apparatus or a user""s taste, even a less important object is saved from being reproduced, a motion picture can be still identified.
In order to encode a picture having an arbitrary shape (i.e., an object), an appropriate transformation method adapted to the shape is employed, such as the xe2x80x9cshape adaptive discrete cosine transformxe2x80x9d, or an insignificant region of the picture is padded by a predetermined method and then a conventional cosine transform (8xc3x978) is provided, where the insignificant region is an outside of the display region of the object, and contains no pixel data for displaying an object, in other words, the region consists of insignificant sample values only. On the other hand, insignificant sample values can be found at the object boundary of a prediction region (e.g., a block consisting of 16xc3x9716 pixels) which is obtained through a motion compensation of a reference picture reproduced in the past for removing a redundant signal between pictures. This type of prediction region is firstly padded, then a the difference between the subject region and the predict region is obtained, and then, transformed and encoded. The reason why the prediction region is padded is to suppress a differential signal.
When the efficiency of encoding/decoding a digital picture is considered, how to pad the insignificant pixels is an important subject, and this influences a decoded picture quality and transmitting data quantity.
The prior art discussed above discloses the following steps: An overall picture is referenced and padded first, to prevent a prediction region from including insignificant sample values, then the prediction region is obtained by a motion compensation or other methods. How to pad the overall picture is, repeating a significant sample value on an object boundary and replacing an insignificant sample values therewith. When a sample is padded by scanning both horizontal and vertical directions, an average of both the padded values are taken. This conventional method pads the whole picture, and therefore providing a prediction region with less errors for a picture having a great motion.
However, when the whole image of a reproduced reference picture is referenced and padded, the reference picture must be entirely decoded, before padding can be started. When repetitive padding is applied, the amount of calculation increases in proportion to the picture size. In other words, this padding method requires a large amount of processing and a long delay time, and sometimes results in very large amount of calculation, for reproducing a picture.
In order to avoid calculation proportional to the picture size, a reproduced boundary region should be padded on per-region basis. This method can solve the delay time and volumes of calculation. However, since this method pads only the boundary region, the significant regions are limited within the internal region bounded by the boundary regions, and hence limiting the effect of padding. Therefore, this method cannot produce a prediction signal having less errors for a motion picture with a great motion.
Since the method of padding the overall picture results in increasing data amount, only a small advantage can be expected. In other words, an insignificant pixel has no pixel values to be encoded, and when significant pixels are encoded together with an insignificant pixel, coding efficiency is lowered. For example, when the significant pixels are all in black, the coding efficiency is lowered if insignificant pixels are in white, on the other hand, the coding efficiency is promoted if the insignificant pixels are in black. As such, a value of the insignificant pixel does not influence a quality of a reproduced picture, but influences the coding efficiency, therefore, how to deal with the insignificant pixel value should have been discussed with care.
The present invention aims to, firstly, provide a padding method, through which a prediction signal with less errors can be produced for a motion picture having a great motion, accompanying a short delay time and a small volume of calculation.
In order to achieve the above goal, according to the present invention, in a digital picture data including picture information which indicates an object, a picture is resolved into a plurality of regions adjoining with each other, and insignificant sample value of a region containing the boundary of the object shape is padded by the values obtained from transforming the significant pixel values near to the insignificant pixel values.
The simplest functional transformation is that an insignificant pixel value is replaced with a significant pixel value adjoining thereto, and this replacement is just repeated. The combination of this repetitive replacement method and the above method can produce the more effective padding.
Further, there is a method of enlarging a padding region to an appropriate extent. This method extends the region to be padded to an insignificant regions consisting of insignificant pixel values only, where the insignificant regions are near to the regions containing an object boundary. In addition to padding these insignificant regions, this method also pads the regions containing the object boundary using values obtained by applying a functional transformation to the significant pixel values of the region. This method enables processing involving larger motion compensation.
The present invention aims to, secondly, apply the above method of padding a digital picture to the methods of encoding/decoding digital picture and the apparatus thereof, whereby a picture compression process producing the better picture quality with a small amount of processing data can be realized.
In order to achieve the above goal, a picture encoder comprising the following elements is prepared: In a digital picture data including picture information which indicates an object of the input signal, where the input signal comprises (1) a signal indicating a pixel value and (2) a significant signal indicating whether a pixel value of each pixel is significant or not, the picture encoder comprises,
(a) predicted picture generation means for producing a predicted picture signal corresponding to the input signal by using a decoded picture signal,
(b) pixel value generation means for resolving the picture into a plurality of regions adjoining to each other, padding the insignificant sample value of the region containing a boundary of the object shape with a functional-transformed significant pixel values located near to the above insignificant pixel value,
(c) subtraction means for subtracting the output of the predicted picture generation means from an output of the pixel value generation means,
(d) encoding means for encoding the output of the subtraction means,
(e) decoding means for decoding the output of the encoding means,
(f) adding means for adding an output of the decoding means and the output of the predicted picture generation means, and
(g) memory means for storing the output of the adding means temporarily for further use in the predicted picture generation means,
wherein the output of the encoding means is an output of this picture encoder.
The corresponding digital picture decoder comprising the following elements is also prepared:
(axe2x80x2) decoding means for decoding the input signal,
(bxe2x80x2) predicted picture generation means for producing a predicted picture signal corresponding to the input signal by using a decoded picture signal,
(cxe2x80x2) pixel value generation means for producing a pixel value from significant pixel value in the predicted picture signal by using a predetermined function, replacing insignificant pixel value of the predicted picture signal with the produced picture value, and outputting the replaced pixel value,
(dxe2x80x2) adding means for adding an output of the decoding means and an output of the pixel value generation means, and
(exe2x80x2) memory means for storing temporarily an output of the adding means for further use in the predicted picture generation means,
wherein the output of the decoding means is an output of this picture decoder.
An insignificant region adjoining to the boundary of object shape and consisting of insignificant sample values only, is padded, whereby processing region is appropriately enlarged without increasing data volume remarkably, and as a result, the accuracy of processes including a motion compensation is promoted.
To be more specific about the padding method of a digital picture according to the present invention, the method comprising the following steps is prepared:
a first padding process for scanning a picture sample having an arbitrary shape consisting of significant and insignificant sample values along a first direction, and in the first direction, producing a first padded picture by replacing the insignificant sample values with the significant sample values selected through a predetermined method,
a second padding process for scanning each sample of the first padded picture consisting of significant and insignificant sample values along a second direction, and in the second direction, replacing the insignificant sample values of the first padded picture with the significant sample values selected through a predetermined method or the sample values padded in the first padding process.
To be more specific about the padding method of a digital picture according to the present invention, another method comprising the following steps is prepared:
resolving a digital picture having an arbitrary shape into a plurality of regions,
processing the regions according to a predetermined order,
padding the insignificant region adjoining to a boundary region at the shape boundary and consisting of insignificant sample values only, with a predetermined padding values.
When the subject region is not an insignificant region, in particular, if a previous region adjoining to a subject region is an insignificant region at the predetermined order, the previous region is padded with a padding value found through a predetermined method.
When the subject region is an insignificant region, in particular, if a previous region adjoining to a subject region is not an insignificant region at the predetermined order, the subject region is padded with a padding value found through a predetermined method.
A picture encoder employing the method of padding a digital picture according to the present invention comprises the following elements:
input means for receiving a digital picture data having an arbitrary shape,
process means for resolving the digital picture into a plurality of regions adjoining to each other,
a first adding device for receiving a data of a subject region and a data of a prediction region, and producing a data of a differential region,
an encoding device for receiving the data of the differential region, and compressing thereof into a data of a compressed differential region through a predetermined method,
a decoding device for receiving the data of the compresssed differential region, and decoding thereof into a data of an expanded differential region,
a second adding device for receiving the data of the expanded differential region, adding the data of the prediction region thereto, and producing a data of a reproduced region,
a first padding device for receiving the data of the reproduced region and padding the insignificant sample values included in the reproduced region through the previously described padding method,
a frame memory for storing the data of the reproduced region of which insignificant sample value has been padded.
Instead of or in addition to the first padding device, a second padding device is employed for padding insignificant sample values included in the prediction region.
A picture decoder employing the method of padding a digital picture according to the present invention comprises the following elements:
input means for receiving a compressed coded data,
a data analyzing device for analyzing the compressed coded data, and outputting a compressed differential signal,
a decoding device for decoding the compressed differential signal into an expanded differential signal,
an adding device for adding the expanded differential signal and a prediction signal, producing a reproduced signal and outputting thereof,
a first padding device for padding an insignificant sample values included in the reproduced signal through the previously described method,
a frame memory for storing a picture data padded by the first padding device as the prediction signal.
Instead of or in addition to the first padding device, a second padding device is employed for padding insignificant sample values included in the prediction region.